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ABSTRACT 


The Critical Task, previously described in Part I of this report, 
consists of a closed- loop tracking task in which an increasingly unstable 
controlled element is used to yield a measure of the operator's effective 
time delay while tracking. In this part of the report, the task has been 
further developed through the analysis of additional, data. 

In a series of experiments, it was found that the human operator's 
characteristics do not change as the system input level is decreased; 
hence, the critical task yields a valid limit when excited solely by 
the operator's remnant. The effects on the operator of different control 
stick types (force, spring, and free) are investigated, and the differences 
in critical task scores are related to the operator's describing function 
characteristics. Step reaction time data axe compared with the continuous 
measures of effective time delay and the autopacer scores. A sample analy- 
sis to determine the number of autopacer trials necessary to achieve a 
confident measure is made. Finally, data are presented for a second- 
order critical task in which an integrator precedes the first-order 
divergence . 
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SECTION I 


INTRODUCTION 


A closed-loop compensatory tracking has "been developed which uses 
an increasingly unstable controlled element to yield a measure of the 
operator's effective delay time while tracking. The main body of data 
and the theoretical analysis of the task is reported in Part I of this 
report (ref. l). The experimental program for this research was under- 
taken in February 19&5 at the Franklin Institute where, as part of the 
over-all experimental plan, some data were obtained which were rather 
incidental to the main program but which could not be obtained economi- 
cally at a later date. It is these data that are reported herein, with 
the purpose in mind of further advancing the critical task theory, under- 
standing the observed behavior of the operator, and validating some of 
the theories which have been evolved. It is assumed throughout this 
report that the reader is completely familiar with the companion Part I 
report (ref. 1), so that some details are not repeated here. 

The data obtained cover several areas of interest. Each area is 
essentially independent, hence this report has been written in the form 
of a collection of experiments. Section II covers the effects of varying 
input level on the operator to validate the use of the autopacer without 
an input. Section III contains the effects of different control sticks 
on the results. Section IV presents concurrently measured step reaction 
time data and an analysis of trial size, while Part V covers some pre- 
liminary second-order critical task data. 


1 


SECTION II 

INPOT EFFECTS ON OPERATOR CHARACTERISTICS 


A • GENERAL 

When the controlled element of a manual control system is unstable, 
the operator must exercise control even in the absence of an input, as 
would any linear controller. A mechanical or electrical controller 
would have no noticeable output (control would be accomplished with 
very small motions) while the human operator exhibits varying degrees 
of control action depending on the level of instability. An explanation 
is that the operator's remnant (sometimes attributed to "noise" injection 
or time variation about mean parameters, ref. 2) excites the system and 
is, in effect, a system input. The STI critical task, which utilizes an 
unstable plant, was developed not as a tracking task, per se, but as a 
device to measure certain operator characteristics while in closed-loop 
operation. Thus, if these operator characteristics could be adequately 
measured using only remnant for system excitation, economies in mechani- 
zation and operation would result . The experiment described below was 
carried out to determine the nature and assess the validity of the effec- 
tive time-delay measures obtained as the input level is progressively 
reduced to zero. 

B. DESCRIPTION COT THE EXPERIMENT 

The experiment was carried out at the Franklin Institute simultaneously 
with the experiments described in reference 1 and used the same simulation 
and analyzers to take data for describing function and remnant calcula- 
tions. As will be recalled from reference 1 , the system input consisted 
of the sum of ten sinusoids with their amplitude adjusted so that the 
spectrum was rectangularly shaped and had a higher frequency, lower 
amplitude "shelf" as shown in figure 1 . 

The experimental variables of the input were chosen to be the band- 
width, aij_, and the mean- square amplitude, given by 
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where Aj is the peak amplitude of the jth sinusoid. Random phasing 
between the components gives a random-appearing input signal. 



w( rad /sec) 


Figure 1. Input Spectrum (b6 ' shape, = 1-5 rad/sec) 

The Measurement Systems, Inc. Model 455 force stick was used for all 
runs and the first-order sub-critical task of reference 1 was used with 
X = 4 rad/sec (X is the unstable root value) . The subject was the same 
in all the Franklin Institute critical task runs. Approximately a 
dozen runs were made specifically to determine input effects. Four 
input levels and three input bandwidths were used, and two no -input 
runs were tape recorded for further power-spectral analysis by the dc 
analyzer . 

C. CRITERION FOR VALIDITY ADD EXPERIMENTAL DESIGN 

The approach for determining the validity of no-input measures is 
as follows. 

Consider the typical manual control system shown in figure 2. In 
the absence of a system input, I, any attempt to measure the linear 
characteristics, Yp,. of the operator would result in 

v = £. = _J_ (2) 

X p E Y c 
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unless a specific form is assumed for N c , the operator's injected noise 
or remnant. Let us assume for the moment that the linear description 
of the operator does not change in the absence of an input, i.e., that 
for Y c = 1 / (— Ts + 1) it is given by the "extended crossover" model, 

Y p - K p e (a / s - Ts) (3) 

to an adequate approximation (this is shown to be an adequate model in 
ref. 1 when a system input is present). The characteristic equation of 
figure 2 is unchanged in the absence of an input so the stability cri- 
terion is 'unchanged. We can therefore show that a critical instability, 
\ c , obtained without an input is valid if we can show that the form of 
Yp doesn't change when the input is progressively reduced to zero. 



The experimental design to permit this extrapolation to zero input 
is as follows: 

1 . A series of inputs having a constant effective band- 
width but decreasing rms levels is given the operator, 
and the resulting describing functions, control output 
power spectra and remnant power spectra are measured 
for each. The describing functions and remnant are 
then extrapolated to zero input level. The extrapo- 
lated data are used to "predict" the control output 
power spectrum that remains when the input is removed, 
and this is compared with the measured control output 
spectrum. 
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2. Since the operator remnant appears to be wide -band 
(ref. 3), a very low amplitude input can be used 
while being subjectively masked, at the error 
display, by the relatively large remnant-excited 
signals present during near-critical tracking. 

This "subliminal" input is nevertheless acted on 
by the operator and permits a measurement of Yp 
under subjectively no-input conditions, albeit 
with degraded signal-to-noise ratios. 

D. DISCUSSION OF THE DATA 


The data obtained from the watthour meter and dc cross-spectral 
analyzers (ref. 3) can be reduced to yield results of essentially four 
types: (l ) describing functions, Y (jo), (2) control output power 

spectra, ^ cc (od) , (3) operator's control remnant, ^(ai), and (4) average 
performance measures, c^, e2, and p ac * The first three types indicate 
any frequency sensitive changes occurring within the operator, while 
the fourth category shows how the net closed-loop performance varies. 

All four are useful as clues to behavior, and each will be discussed in 
turn below. 

1 . Describing Function Data 

Operator describing functions have been computed in the manner de- 
scribed in reference 3 for selected runs from watthour meter analyzer 
data taken during the experiments. The describing functions, plotted 
in figure 3> are computed for three input levels. The levels were 
chosen on the basis of subjective operator difficulty and are considered 
to be representative of "small, medium, and large" inputs. 

The most notable difference between input conditions apparent in 
figure 3 is that the low amplitude input data is scattered.* This 
emphasizes that the analyzer has difficulty resolving measures at a 
frequency where only a few cycles of the input occur in a run length. 


*Machine problems prevented data recording at 13*8 r/s on the day 
the l/l6" and 1 /4" rms runs were made, hence the absence of describing 
function points. 
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The difficulty is compounded hy the low signal -to-noise environment of 
the low amplitude input. The high frequency portion is reliable, however, 
and careful inspection of the high frequency portion of the describing 
function phase yields no trends or significant effects from input level 
variation. The appropriate curve fit from reference 1 (a = 0.35 rad/sec, 
T e = 0.110 sec) is shown in figure 3* These results are consistent 
with the work of Elkind (see ref. 4) where it was determined that system 
closed-loop characteristics remained the same as the input was dimi ni shed. 
The amplitude ratio data also indicates that little change occurs, but 
has not been discussed since it is less sensitive than the phase to system 
changes (note that the operator model, eq. (3), has only one parameter, 

Kp, that affects the amplitude ratio) . 

Additional describing function data were taken which allow a check 
of input effect by varying bandwidth. Figure 4 shows three bandwidth 
cases (shelf only, o>j_ = 1 .5 and 4.0 rad/sec). The extended crossover 
model fit is shown here also (a = 0.35 rad/sec, x Q = 0.110 sec) and 
again no difference is noticeable in the high frequency character of 
the describing functions. It is interesting to note that reference 3 
did obtain an effect on T e ff from bandwidth variation, but there the 
operators were not operating as near their stability limits, and were 
therefore not constrained to use their minimum T eff . 

It is concluded, then, that at near critical instabilities, no 
further change occurs in the operator's describing function as the 
input is reduced either by decreasing the amplitude or the bandwidth. 

2. Correlated and Remnant Power Spectra 

The describing function shows the operator's linear characteristics 
at input frequencies, but the corresponding power levels at which he is 
operating (determined by closed-loop characteristics) are not obvious 
at all, nor can any information be obtained about his output at other 
than input frequencies. For these reasons, power spectra nicely supple- 
ment describing functions and have been computed for pertinent runs here. 
The methods and procedures used to compute spectra from both the dc and 
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watthour meter analyzers are rather lengthy and so will not he noted 
here (see refs. 3 and 5 for details). 

Since the input is composed of a sum of sinusoids, the portion of 
the operator's output which is correlated with the input is also a sum 
of sinusoids. The power of these sinusoids is represented by the symbol, 
$ccp which is the output power, cc, correlated with the input, i. On 
the other hand, at frequencies between input frequencies, there is evi- 
dence that the operator generates an output over all frequencies (ref. 3) • 
It is therefore appropriate to represent the operator's uncorrelated 
output (his remnant) as a density, cp^. Consider now the measurement 
of the power spectra at an input frequency with the dc analyzer. The 
analyzer has a finite bandwidth, An, so that it is not only measuring 
the correlated power, but the remnant within Ad. The power measured 
will thus be the sum of the power in the sinusoid ("spike" power) and 
the area under the curve in sketch A. The techniques for separating 


Output 

Amplitude 



Sketch A. Measurement Bandwidth of the DCA 

this resultant measure into $ cc ^ (the squared amplitude of the "spike") 
and 9nn ("^hc amplitude of the crosshatched area of the curve) are contained 
elsewhere (ref. 3) , but now the reader will not be perplexed at finding 
remnant at input frequencies in some of the following figures. 

Before proceeding with the data presentation, it is interesting to 
compare the results of the two possible measurement methods. Figure 5 
presents the average power spectra for 5 runs obtained from the watthour 
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Figure 5* Comparison of DC Analyzer and Watthour Meter Analyzer Data 
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meter analyzer and the dc analyzer (the same 5 runs were used for each) . 
There it is apparent that the agreement is excellent "between the inde- 
pendent and entirely different techniques. 


The three conditions of interest are: (l) the base condition with 

the B6' — 1 .5 — 1 /8" input, (2) the shelf input condition, cTj_ = 0.035"> 
and (3) no input at all. Figures 6 and 7 show the measured correlated 
power, $ cc ^, for the base condition and the shelf input condition, respect- 
ively. The solid line in each figure is the theoretical spectra obtained 
from 


■cc-i 


+ Vc 


r ll 


= ^Gii*ii 


oo 


where the extended crossover model (see fig. 4, ref. l) is used for Yp. 

A comparison of figures 6 and 7 shows that even though the low 
frequency portions of the two are different by l4 dB, the operator's 
linear description remains unchanged, as evidenced by the agreement 
between the data points and the solid lines. 
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— <^> cc Computed from Y p Extended Crossover Model 

O </> cc . Subject Power Output Correlated With Input 
(from DC Analyzer , N = 5) 

B6 - I.5-I/8”, X = 4 rad/sec , Subject: JDM 
OdB = I in 2 

I I 


) 1-0 <j(rad/sec) 100 

Figure 6. Comparison of Base Run Power Spectra with the Model 



<f> cc Computed from Y p Extended Crossover Model 

□ <£ CC| Subject Power Output Correlated With Input 

(from DC Analyzer ; N = I) 

Shelf Input , o-j = .0353”, X= 4 , Subject: JDM 


0 dB = I in 2 






It is also interesting to note that the shelf portion of the input 
is barely detectable by the operator, and yet the power he expends over 
these frequencies (2 < oo < 20 rad/sec) is considerable. 

The above data indicates that the operator doesn't change his form 
as the input is decreased. We can get a further indication of his be- 
havior by measuring power output when no input is present. In such a 
case, his entire output is remnant, and it can be compared with the 
remnant present in the input cases. Figure 8 shows such a comparison. 
Measures were made at input frequencies as well as each half-rad/sec 
between 3 and 10 rad/sec. 



Figure 8. Comparison of Remnant Power Spectral Density for 
Base Runs, Shelf Input, and No-Input Cases 

It is apparent from the figure that the remnant doesn't vary appre- 
ciably as the input is varied from none at all to one of considerable 
size . 

3* Average Measures 

The watthour meter analyzer affords average measures of subject output 
(both correlated and uncorrelated) with relative ease. Thus measures are 
available which provide a crosscheck on the power spectra and which are a 
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good indication of over-all behavior and performance. The most enlightening 
of these measures are the operator's total rms output and his uncorrelated 
rms output (fig. 9) • As is the case with the power spectra, the total 
output increases with input level, while the remnant remains of the same 
order of magnitude. In fact, it would probably be safe to say that the 
rms remnant is essentially constant. 

o ’ j (in) 


1/16 1/8 1/4 



Figure 9* Total and Remnant Average Output Variation 

With Input Level 


is defined as 



1 - 
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2 
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c? + c 2 
c i c n 


(5) 
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and is plotted in figure 10 from watthour meter data. The theoretical 
p a could "be computed from equation ( 5 ), assuming that the remnant c n 
is constant, and compared with the experimentally obtained values in 
figure 10 , but it is felt that the data are too sparse at this time. 

The conclusion, however, that the operator's remnant is a fairly constant 
part of his output is evident. 

X = 4 rad/sec , Subject : JDM 
O cjj = 1.5 rad/sec 
A Wj = 4.0 rad/sec 
□ Shelf Only 



Figure 10. Relative Remnant Variation with 
Input Level 

E. ADTCPACER ASSESSMENT 

It is apparent from the discussion to this point that the operator 
retains his form of behavior in the absence of an input. But a difference 
in autopacer critical instability scores obtained with and without an 
input has been observed, so what is the cause of the difference? The 
answer was given in references 1, 6, and 7 . In reference 1 it was noted 
that as the system approaches instability, the ratio of error to input 



grows until, at neutral stability, the error is infinite. A typical 
error performance curve is sketched in figure 1 1 for a system where 
some optimum strategy, such as maximum gain margin, is being used. 

There, for a given input, the resultant X c is the value obtained when 
the error reaches the display limits. Now, if the input size is increased, 
the "scope limit" line on the figure would effectively be moved down, 
hence X c would decrease. The ideal is to have the display limit very 
much greater than the rms input or effective remnant. 



Figure 1 1 . Characteristic Error Performance for Critical Task 

This phenomenon was checked by mechanizing an analog subject on the 
computer and then varying the input level. The results, documented in 
reference 6, are shown in figure 12. Note that for zero input the 
resulting critical time constant is equal to the delay set in on the 
analog pilot. As the input increases, so does the critical time con- 
stant, i.e., X c (= l/T c ) decreases. This curve, which is supported by 
a theoretical analysis based on input amplitude probability distributions 
in reference 7 , is not intended as a calibration of any sort, since such 
factors as input frequency content, the terminal autopacer rates, operator 
remnant levels, etc., all can slightly influence the resulting scores. It 
does show, however, that the minimum input yields the score most nearly 
equal to the desired critical limit. 


15 


Measured Critical Time Constant , T c ~sec 



Figure 12. Effect of Random Input on Measured T c 
Using a Known and Fixed Value 
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F. SUMMARY AMD CONCLUSIONS 


The effects of an input on the operator under near-critical conditions 
can be summarized as follows: 

• The operator's describing function is pushed to a 
limit and shows no further change with input ampli- 
tude or bandwidth. 

• The operator's output power spectra show that the 
part of his output correlated with the input de- 
creases with input level as predicted with the 
model. 

• The power and spectral shape of the operator's 
output remnant remains essentially unchanged by 
input level. 

It can therefore be concluded that near the critical task limi ts 
the operator behaves as a fixed-linear operator plus noise, irrespective 
of the input level. In the absence of an input, the injected noise can 
be treated as if it were a system input. 


Because of display limits, the autopacer yields the score most nearly 
equal to the true limiting value when the system input is reduced and the 
excitation is due to operator remnant alone. 
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SECTION in 


MANIPULATOR EFFECTS 


A. PURPOSE 

When mechanizing the autopaced critical tracking task, the experi- 
menter must provide an appropriate control stick for the operator. 

Since different types of sticks have resulted in describing function 
differences in past experiments (ref. 1 3) , it was expected that critical 
task scores would also reflect differences due to the manipulator. For 
this reason, an experiment was carried out to determine manipulator 
effects and, ultimately, to establish the suitability of the three 
commonly used types: pressure (isometric), spring restrained, and free 

(isotonic) sticks. 

B. THE EXPERIMENT 

The experiment was carried out simultaneously with those of reference 1 
and used the same measurement equipment and subject. Describing func- 
tion and autopaced data were taken for the three types of control sticks 
available. The force stick, which was used for the bulk of the reference 1 
experiments, was a Measurement Systems, Inc. Wo. 4 35 pencil stick approxi- 
mately 9*2 cm (3-5/8 in.) long. It was mounted on an arm rest at the 
subject's right side and was used longitudinally (fore and aft) for com- 
pensatory tracking in the pitch axis. The free stick, consisting of 
a light aluminum rod attached to a low-torque potentiometer, was 
approximately 5*4 cm (2-1 /8 in.) long and was mounted like the force 
stick. It had extremely small inertia and friction, which could not 
be detected by the operator. For the spring stick portion of the experi- 
ment, a spring was added to the free stick giving a gradient of 24.5 N/rad 
(1.54 oz/deg) . 
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C. RELEVANT NEUROMUSCULAR SYSTEM CHARACTERISTICS 


Before discussing the data, a brief summary of a current neuromuscular 
model will be presented to orient the reader. It will be reca ll ed from 
reference 1 that the effective time delay, T e , is composed of three parts: 
the basic transport delay, equalization time constants, and neuromuscular 
system lags. The transport delay, t^, includes nerve conduction times 
and computational time within the central nervous system, and is there- 
fore invariant with control stick parameters. The lead and lag time 
constants, Tp and Tj, are operator equalization parameters and are 
varied within limits as a function of the dynamic characteristics of the 
controlled element, and so are also invariant with the control stick. 

The neuromuscular system dynamics, then, are the only possible contributors 
to the effective time delay which can vary with manipulator, and will be 
discussed briefly here. 

The dependence of the operator's phase lag (which determines the 
effective time delay) on the neuromuscular system can be seen by con- 
sidering one current model (from ref. 1 4) . A highly simplified block 
diagram is shown in figure 13* There, the system receives a command, 
a c , from the central nervous system and a limb position, 5, results. 


Muscle and Load 



Figure 1 3. Block Diagram of Neuromuscular System 


A high gain and a low gain loop closure of this system are both shown 
in figure 1 La. The total open-loop Bode amplitude asymptotes are shown 
(solid line) along with segments of the open-loop cr-Bode, where, by substi- 
tuting s = -a instead of s = Jcd in the transfer function, the behavior of 
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closed- loop roots on the real axis can he easily determined (see ref. 8 for 
details) . The closed-loop roots are found by constructing the closed-loop 
Bode, Y(jx, = GH/( I + GH), which is shown by the dashed line. For contrast, 
a very low gain closure (dotted line) is also shown. Note the differences 
of the closed-loop roots, 1 /T^, 1 /T Ni , and <%, between the two closures. 

Figure l4b shows the transfer function of interest, 6/c^, obtained 
by dividing the feedback quantities out of the closed-loop plot of 
figure l4a [s/o^ = G/( 1 + GH)]. There the effects of the two gains are 
quite apparent, particularly in the phase. The increase in gain appears 
as a shift to the right of the phase curve. The measurement region shown 
in figure l4 spans approximately two decades.* 


The contribution of the system of figure 1 4 to the operator dynamics 
is then 


_5_ 

«c 


k a 

1 + k a k k 


(Tgs + 1) 


(T^s + 1 ) (%, s + 1 ) 


M 


oft 


+ 1 


(6) 


As noted in reference 1 , the low frequency portion of the phase can be 
approximated by an exponential term, or 




OcLw 4 = S5(4"i) = ~5 (7) 


= -«ja/<n 


a/s 


Recalling that e is a special case of e , the neuromuscular dynamics, 

■which are commonly written in complex plane notation, are then approximately 

a/s 

(8) 
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“c 


V 


( Tt\t^ s + 1 ) 

' « i 


/JL^ 2 + 2 ^ s 
LH/ % 


*il 


*Pract i cally , data to support the model at the low frequency end of 
the band is difficult to obtain accurately due to run length effects, 
which place theoretical limitations on the accuracy of any spectral 
analysis and which can be offset by an increased number of runs . The 
number of runs are limited in this experiment, consequently, the high 
frequency data will be the most reliable. 
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The effects of task difficulty and manipulator differences on the closed- 
loop model parameters can now be assessed. If the operator "tightens up" 
due to task difficulty by raising his gain, K^, the phase curve will slide 
to the right with a resulting decreased Tj^ and increased a and o^. This 
"tightening up" phenomenon has been observed in other work (refs. 3 and 15) 
and was noted subjectively in the experiments of this program. 

To assess manipulator effects, consider changing from a free stick to 
a force stick. The natural frequency, o^, of the muscle and its load would 
be expected to increase as the inertia of the system decreased; consequently, 
an increase in the closed-loop root, o^, would be expected. The extent to 
which the model describes the real situation of the critical task will be 
determined from the data. 

D. DESCRIBING FUNCTION DATA 

The describing function data are shown in figure 1 5* Force and free 
stick data are shown together with spring stick data from reference 3 
for a controlled element divergence of X = 2 rad/sec. Also shown are 
data for the force and spring sticks for X = 4 rad/sec (the spring stick 
of ref. 3 was laterally operated with a gradient of 35*2 N/rad, but in 
this experiment the longitudinal axis was used) . A summary of the experi- 
mental variables appears in Table I. 

Although the amplitude ratio can give some clues to behavior, the 
phase is of primary interest here because we are looking for effects on 
T e . Before looking closely at the high frequency phase, some general 
observations can be made: 

1 . In the crossover region (2 < u> < 15) the amplitude 
ratios for X = 2 rad/sec are nearly coincident, as 
are those for X = 4 rad/sec. 

2. The usual increase in scatter is noticeable at the 
lower frequencies. The scatter generally averages 
out with a few runs, but here the available data is 
limited. Therefore, less emphasis can be placed on 

the low frequency data than would otherwise be desirable. 

3. The high frequency phase shows surprisingly few 
differences due to the manipulator. This portion 
of the describing function will be scrutinized 
carefully after the data is curve fitted. 
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The phase portion of the extended crossover model fits to the describing 
functions are shown in figure 15* The model is given by 

Y p = K p e“^ a / C0+ (9) 

where the a accounts for the low frequency phase "droop" and the x e fits 
the high frequency lag. The T 0 is, by definition, the average effective 
time delay and is the parameter that the autopacer attempts to measure. 

The rules for fitting are to fit the t on the frequency points above 
crossover, which are considered to be highly reliable and repeatable 
points, and to fit the a on the points below the crossover frequency. 

The low frequency points are less reliable since there are fewer cycles 
in a computation period (run length). Past experience (ref. 3) has shown 
that several runs often need to be averaged before a stable a curve develops, 
so it is not surprising that low frequency scatter exists. 

The results of the carve fits are given in table I (to avoid clutter in 
fig. 1 only the curve fit for the force stick with X = 2 is shown) . It is 
seen there that the force stick results in the minimum effective delay time, 
and that the x e obtained from the spring stick falls between the force and 
the free stick values. These observations are consistent with past data, 
reference 13, and autopaced data to be discussed next. It implies that the 
neuromuscular system tightens up with X and that its natural frequency in- 
creases as the manipulator is changed from a free stick to a force stick. 

The actual change, as exhibited in figure 15, is slight but is considered 
significant . 

S. AUTOPACED DATA 

During the experimental series the subject made autopacer runs with 
the force, spring, and free sticks. The data are shown in figure 16. 

Each data point shown is an average of five trials. The lines connect 
comparable runs made on the same day. The total averages for the entire 
series with and without input are also shown. The differences between 
the daily runs and the total averages reflect the day-to-day variation 
in the five trial means. 

The autopaced data shows the same trends with manipulator as did the 
describing function T e , i.e., the T c = l/X c resulting from the force 
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Figure 1 6 . Autopacer Data for Three Types of Control Sticks 


stick is about 90 percent of the free stick value. The spring stick auto- 
paced data results in a rather large T c . This anomaly cannot he resolved 
since only five trials were made with the spring stick. However, it is 
noted that the trend for that day was consistent with the other day's 
trends and also with the describing function trends. 

A comparison of the fitted T e 's (from Table i) from the describing 
functions and autopacer T c ' s obtained without an input is made directly 
in the regression diagram of figure 17. The = 0.75 T c line appears 
to be the best fit to the data. 

7. REMNANT DATA 


The relative amount of power in the operator's output which is 
correlated with the input is called the relative remnant and is defined 
as 



where 


n = uncorrelated signal in 
the operator's output 

c = total operator's output 
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Figure 1 7* Regression Diagram for Describing Function 
Fitted T e and Autopacer T c 

The remnant was computed from available spectral analyzer data and is 
shown in figure 18 for the describing function runs of table I. The 
reference 3 spring stick remnant data for X = 2 is averaged data of four 
operators. It does tie in satisfactorily, however, with the data of 
this experiment. 

The remnant data shows that the operator becomes increasingly "noisy" 
when going from a force stick to a free stick. The reasons for this are 
not clear, but similar observations have been made elsewhere (ref. 1 3 ) • 

It is most likely that both force level and displacement level influence 
remnant behavior to some extent. 

G. SUMMARY AND CONCLUSIONS 

The describing function and autopacer data clearly indicate that the 
minimum effective time delay is exhibited by an operator using a force 
stick, while sticks requiring hand or finger displacement lower the 
natural frequencies of the stick-arm combination, causing an increase 
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Figure 1 8 . Relative Remnant for Three Types of Control Sticks 

(only slight for the sticks tested) in the operator's effective time 
delay. The data further indicates, when considered with the remnant data, 
that there is no essential change in the operator's mode of behavior with 
manipulator. We can therefore draw conclusions about the two common 
applications and the results can be extrapolated to other situations as 
required by the experimenter. 

One of the likeliest applications of the critical task is to measure 
the crucial parameter, t e , of the operator that he would exhibit while 
tracking in a specific vehicle. The parameter then is a measure of the 
operator's capability and proficiency. Since stick differences are 
reflected in the measure, it should be made with a stick resembling as 
closely as possible the one in the actual tracking situation. 

On the other hand, if the experimenter is attempting to determine the 
effects of stressors, environment, etc., on operator tracking behavior, 
performance, and characteristics, it is the perturbations about the base 
values that are of interest. The experimenter can therefore use the stick 
of his choice as long as he keeps in mind that the perturbation effects 
are not necessarily transferable between stick types. 
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SECTION IV 


STEP REACTION TIME DATA AND TRIAL SIZE DETERMINATION 

A. STEP REACTION TIME DATA 
1 • General 

The critical task measures the effective time delay of an operator, 
which is the continuous tracking analogy of the discrete measure step 
reaction time delay. Because of this analogy, it is of interest to 
directly compare the two measures since any correlation "between them 
has never really been established. For this reason, some step reaction 
time measures were made; the resulting data and comparisons with effec- 
tive time delay are given here. 

2 * The Experiment 

The step reaction time data were taken during the experimental program 
when the describing function and autopaced runs were made at the Franklin 
Institute. A relay- controlled integrator was used to measure the time 
lapse between stimulus and response. The display and control stick were 
identical to those used in the tracking tasks, i.e., a horizontal bar on 
the CRT which displaced vertically, and a force stick was used. The 
subject was instructed to quickly return the bar to the crosshairs by 
applying pressure to the force stick when the bar unexpectedly displaced. 
Twenty responses were obtained per run, and the runs were integrated into 
the experimental plan so that they occurred adjacent to an autopaced run. 

Two types of reaction time, ET, were measured: simple , where one polarity 

of stimulus was applied; and dis junctive , where the bar could displace in 
either direction. Both the direction of the displacement, when disjunctive 
HT was measured, and the foreperiod of the reaction (the time lapse between 
the warning signal and the stimulus, 0 < tf < 5 sec) were random. 


29 



3. Data 


The data is shown in figure 19. The reciprocal of the run means 
rather than the means themselves are plotted for two reasons: (l) the 

reciprocal is directly comparable with the X scores (in rad/sec) obtained 
from the autopacer, and (2) there is evidence that 1 /KT is normally dis- 
tributed (ref. 9) , which implies that nerve conduction velocity may have 
a Gaussian distribution about its mean. It can be noted in the figure 
that the deviation is considerable; the average ratio of standard devia- 
tion to mean is 21 percent. The ratio obtainable under the best of 
conditions is 10— 1 3 percent (ref. 10, p. 38), so that the measures here 
are considered to be typical BT measures. 

A comparison of step reaction time delay with either effective delay 
time or critical task score (X c ) is a comparison of two different types 
of measures. The reaction time delay is a discrete measure with a very 
limited number of degrees of freedom (approximately equal to the number 
of trials) , whereas the describing function and critical task measures 
are continuous measures and contain a considerably higher number of 
degrees of freedom (proportional to bandwidth times run length) . We 
would therefore expect a higher variability in the step reaction time 
delay, which is indeed the case (the ratio of standard deviation to mean 
for the autopacer scores is approximately 6.5 percent for five trial 
samples) . 

Since the three kinds of measures were made at essentially the same 
psychological and physiological times, and with the same display and 
manipulator, a regression diagram could be drawn to indicate the connec- 
tion, if any, between the measures. There would, however, be a degree 
of arbitrariness in "pairing off" the adjacent measures. The behavior 
of the sample means are therefore compared by normalizing them (dividing 
by the mean for the total experiment) and plotting them in the order of 
occurrence in figure 20. The elapsed time between each successive run 
of an "AM" or "PM" group is approximately 5 min. 
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Figure 19* Run-to-Run Variability of the 20-Trial 
Reaction Time Samples 
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Figure 20. Temporal Trends of Normalized Sample Means 
for Three Types of Measures 



The distribution in figure 20 has a random appearance, i.e., there 
is no noticeable correlation between the variations in the adjacent 
measures for this subject. The variations are due to random variables, 
then, and do not reflect a short term or steady-state change in the 
operator ' s characteristics . 

4. Summary and Conclusions 

Comparison of autopaced scores with step reaction time delay supports 
the conclusions of reference 1 that valid measures are obtainable with as 
few as five trials in a sample. The ratio of the deviations of the mean 
to the over-all mean is approximately 6.5 percent, which compares quite 
favorably with a 21 percent ratio for step reaction time delay for twenty 
trial samples. 

A cursory look at the short term variations of the means indicate 
that they are random, i.e., an increase in an autopaced score sample 
does not imply a corresponding decrease in an immediately following step 
reaction time delay measure. 

B. DETERMINATION OF TRIAL SIZE 

1 . General 

A primary use of the critical task is intended to be the determination 
of the effects of various environmental and stressor factors on critical 
task scores. Since some situations might be difficult to generate over 
long periods of time, a minimum number of trials to establish the effects 
should be predetermined. Here, a demonstrative example will be presented 
to show how such a determination might be made. 

2. Analysis 

It was shown in reference 1 that the critical task score, l c , exhibits 
a normal distribution about the mean for a single operator under what 
might be termed a "normal" environment and set of circumstances. There, 
for the input case, it was shown that the parameters of the distribution 
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are ~ c ^ =6.00 rad/see, s-j = 0.39, n-j = 4-6 trials. The subscript 1 has 
been added to show that the parameters are for "condition 1 , " or the 
normal condition. Now let us assume that a second experiment is to be 
run with the same operator under a different set of conditions. We wish 
to detect any change in the score parameters due to the new environment. 

The four distinct possible outcomes to the experiment are summarized in 
table II (a treatment of each case is given on p. 395 of ref. 11 ). We 
shall only consider one of these (outcome B of table II) in our sample 
calculation. The treatment of the other three cases are as equally 
straightforward (the F-test is used to determine differences in variances) . 
Since data to the contrary is lacking, the assumption that the variances 
are equal for our example is not unreasonable, and it can be easily checked 
when two experimental environments are available. 

TABLE II 

POSSIBLE OUTCOMES OF TWO INDEPENDENT EXPERIMENTS 


OUTCOME 

MEANS 

VARIANCES 

A 

X ci = ^c 2 

_2 2 
0 1 ~ a 2 

B 

^ci ^ ^C2 

C 

= *C 2 

2,2 
C 1 + °s 

D 

^C1 ^ ^C 2 


A simple t-test will be used to determine the minimum trial size in 
the second experiment. Given two sets of independent observations, form 
the sample statistic 


t c 



( 10 ) 


3 ^ 



with n-] + ng — 2 degrees of freedom (ref. 1l). We w ill specify the minimum 
difference of means which we want to detect as |x^ — Xpl = 0.5 rad/sec. 

This is an engineering judgment and may need to he revised as experience 
is gained with different environments. For this example, the sample 
statistic becomes 


t 


c 


1 .28 




0.0218 + 


n 2 


( 11 ) 


which is plotted in figure 21 . 



The sample statistic, t c , is to he compared with the computed statistic, 
t', based on tables of the t-distribution. When the two experiments have 
different sample sizes, a modified form of t' is appropriate. A reasonably 
conservative version (ref. 12) is 


t' 


2 2 

Bf S2 

aj* ta >ni + n 2 t& > n 2 



( 12 ) 
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where t a ^ n is the value found in the tables at the a-level of confidence 
for n — 1 degrees of freedom. 

When s-j = S 2 > as is assumed above, equation (12) reduces to 


t' = 


n 2^a,ni + n -\^a,n2 

n-] + ng 


for the example here. Equation (13) is plotted as a function of n 2 in 
figure 21 for a = 95$ and n-| = 46. The condition indicating a significant 
difference in means of 0.5 requires that 

t' < t Q (l4) 

so it can be seen in the figure that at least six trials are needed. On 
the other hand, little is to be gained by running more than ten, so the 
experimenter would probably specify two samples of five trials each. 

3. Conclusions 

Having established a "base line" set of parameters for an operator 
under normal conditions, the experimenter can easily predict the number 
of trials necessary to detect a change of the mean under different condi- 
tions. The simplifying assumption, that variances remain unchanged, 
requires experimental verification, however, and should be among the 
first questions answered in future programs involving multiple test 
environments . 
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SECTION V 


SECOND-ORDER TASK DATA 


A. GENERAL 

During some of the initial analyses of the first-order critical task 
it "became apparent that the addition of an integrator to the task would 
require the operator to generate large amounts of lead to stabilize the 
system. Such a second-order critical task might then measure not only 
the operator's time delay, as in the case of the first-order task, but 
might also provide an indication of the operator's lead generating 
capability. In order to obtain an appreciation of the magnitude of the 
problems involved in using the second task to yield useful and meaningful 
data, the experiment described below was carried out. The data obtained 
was rather meager, but nevertheless compares with that in reference 3 and 
expands the data base. 

B. THEORY 

The second-order critical task is given by 


Y 


c 


-K C 

s( — Ts + 1 ) 


(15) 


The form adopted by the operator, as shown in reference 3, can be described 
by the extended crossover model, 


Y p = Kp(T L s + l)e + 


a/cn) 


(16) 


where the operator attempts to cancel the integrator of the controlled 
element by using a large amount of lead, T^. The total open-loop expression 
is thus 


y v - K ( T k s 1 1 ] e “d( + 
P c s(— Ts + 1 ) 


( 17 ) 
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Generic root locus and Bode diagrams for the system are almost identical 
in appearance to figure 3 of reference 1 , the exception being an extra 
pair of roots near the origin in the second-order root locus due to the 
lead term and integrator. 

An exact analysis of the stability limits of the second-order task 
would require the solution of higher order equations, which cannot be 
done neatly in literal terms . However, a good approximate solution will 
be given here. 

Appendix A of reference 1 gives for the first-order task at critical 
conditions : 

x c T e = 1 ~ V°^e” 08) 


This result was obtained by first deriving an approximate expression for 
the phase in the crossover region (where the stability characteristics 
are determined) using the extended crossover model, then manipulating 
the phase equation to determine the stability limits . The total open- 
loop expression for the first-order task is 

Ke ~j ( T e U} + co/cd) 


Vc 


-Ts + 1 


09 ) 


where X = 1 /T . 

Comparison of equations (17) and (19) indicates that the difference 
between the phases of the two tasks is given by 


^Pl = * 


(T l s + 1 ) 


— ^ + tan 1 T l o> 


( 20 ) 


which can be approximated in the crossover region, where Tjtn » 1 , as 


A «\ * 3"C . it 

A£Pl = - z + z - 


1 

T l o> 


1 

T L cn 


( 21 ) 
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Note that this contribution to the phase is of the same form as the 
a term in equation (19)* The effect of Tp on X is thus obtained simply 
by replacing the a of equation (l8) by a + 1 /Tl, resulting in 

Ve = 1 - V( a+ l/T L )*e (22) 

Thus, the differences between the theoretically obtainable scores 
for the first- and second-order critical tasks are clearly shown by a 
comparison of equations (18) and (22). The total effect of requiring 
the operator to greatly increase his lead is to reduce X c for two reasons. 
First, the effective time delay, T e , is increased since high frequency 
lead is no longer available to help cancel out high frequency neuromus- 
cular system lags. Second, the phase margin is reduced slightly because 
of the increased low frequency phase "droop" (i.e., an increase in the 
apparent a) . 

The data taken to validate this theoretical effect is given next. 

C. DB80BZBING FUNCTION BATA 

Describing functions were measured for two values of X. Two runs 
were made for X = 1 rad/sec, which was moderately difficult to control 
for four minutes, and five runs were made for X = 2 rad/sec, which was 
nearly at the operator's controllability limit. The b 6' — 1 -5 — 1 /8 in. 
input was used in a.1 1 runs. 

The resulting describing functions are shown in figures 22 and 23- 
Also shown in the figures are the extended crossover model fits. The 
figures axe very similar in general appearance, specifically: 

• Large low frequency phase variation is present, 
due to a small number of runs and/or large 
variability in lead generation. 

• A high frequency phase droop at a considerably 
higher rate than that provided by the model. 

This condition would be improved by fitting with 
a precision model, where the neuromuscular system 
dynamics give a more rapidly lagging phase with 
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k-o 



4 Yp Y c (deg) 





frequency. Since both models are the same in 
the crossover region and since the data are not 
as numerous as might be desired, the precision 
model fit will not be made. 

• Low phase margins (on the order of 10 deg) are 
demonstrated, indicating the difficulty of the 
task. 


A difference of note between the two cases is the high frequency 
amplitude ratio for the X = 2 case. There the peak due to the neuro- 
muscular second-order pair is obvious. Here again, the precision model 
would improve the fit greatly. 

A comparison of the a-model fitting parameters is shown in table III, 
together with the parameters of the second-order task data of reference 3 * 
The effective time delay shows a decrease with increasing task difficulty, 
as it did in the first-order case. There is also a fairly large increase 
in a, which is presently unexplained. The relative remnant, p a , indicates 

TABLE III 

COMPARISON OF SECOND-ORDER FITTED PARAMETERS 


CONTROLLED 


K c 

K c 

V 

ELEMENT 

s(s - 1 ) 

" s(s - 2 ) 


TABLE XIII 

PRESENT 

PRESENT 


REF. 3 

EXPERIMENT 

experiment 

Input 

b 6'-1 .5-1/4 in. 

B6'-1 .5-1 /8 in. 

b 6'-1 . 5—1 /8 in. 

No. of Runs 

12 

2 

5 

0 ^ (rad/sec) 

3.7 

3-5 

5-2 

% (<leg) 

7-0 

15.0 

5-0 

x e (sec) 

0.31 

0.23 

0.19 

a (rad/sec) 

0.37 

0.57 

0.70 

<XT e 

0.12 

0.13 

0.13 

coui (rad/sec) 

1 .2 

1 -3 

2.1 

cuu 2 (rad/sec) 

3-7 

5-22 

6.0 

1 /Tp (rad/sec) 

0.20 

0.20 

0.20 

K p(s 7 i^) 

3-8 

3-6 

2.5 

p a 

0.47* 

0.69 

0.49 


*Average for five subjects. Range: 0.21 < p a < O .56 
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that the over-all signal to noise ratio is lower than in the first- 
order case, and so decreases with task difficulty (p a = 0.^9 vs about 
0-7) which further supports the existence of the low frequency scatter. 

The limited data with its attendant scatter leaves some arbitrariness 
in the low frequency fitting of a and 1 /Tpj consequently, further specula- 
tion on the cause and effect relationship for this data will be curtailed. 
Nevertheless, the describing function results can be compared with the 
autopacer data to follow. 

D. AUTOPACER DATA 

During the experimental program, a limited amount of second-order 
autopaced data were taken to find the effects of lead generation on X c . 
Three samples of five trials each were made with an input, and a like 
number were made without an input. These data are plotted in figure 2b 
where the five trial means are shown by the symbols with the standard 
deviations shown by the bars . The ratio of deviation to mean is 1 0 per- 
cent for the b 6' input case and 11 percent with no input. This is 
approximately twice the ratio observed for the first-order data, but 
still compares quite favorably with the 21 percent ratio observed for 
the step reaction time data (see Section IV-A) . 

In Section V-B a theoretical relation for the autopacer score was 
derived (eq. 22) using the extended crossover model. The theory and 
data can now be compared. Using the values of Te, oc, and Tp from 
table III for the X c = 2 case, equation (22) yields 


X CDF * 0^35 (' - + °- 2 °) °-' 8 5 ) 

= 3.2 rad/sec 


CDF 


(23) 


This computed score is to be compared with the average autopacer limit 
of X c = 3*2 rad/sec from figure 2b. The remarkable agreement here raises 
an interesting question. Why is the operator able to track at such a 



Figure 2b. Second-Order Autopaced Data 

small phase margin in the second-order case (5 deg here vs 12 deg for 
the first-order case, resulting in the accurate prediction of eq. 22) 
even though more variability in the data, and therefore in the operator 
was observed? In the first order results in reference 1 an 1 8 percent 
difference between predicted and observed scores was exhibited because 
the remnant was not included in the analysis . 

Another unresolved observation regards the control stick. In the 
first-order case, the type of stick (force, spring, or free) made little 
difference in behavior (see Section III) . For the second-order case, 
however, the operator was unable to control even the smallest instability 
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with the free stick. His subjective impression was that lack of a null- 
control reference was the difficulty. This difference between the first - 
and second-order free -stick critical tasks will hopefully be explained 
by one of the comprehensive neuromuscular system theories currently being 
developed. 

E. CONCLUSIONS 

The operator behaves as predicted when tracking the second-order 
critical task. A remarkable, perhaps fortuitously good, agreement between 
the theoretical limit implied by the describing function data and the auto- 
paced limits was noted. The adjustment rules of the operator, however, 
are not clear, and will have to be the subject of future work. Because 
of this, and because of the relatively scant amount of second-order data 
available, it is concluded that the second-order critical task requires 
additional research before it can become as useful a tool as the first- 
order task. 
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“The aeronautical and space activities of the United States shall he 
conducted so as to contribute ... to the expansion of human knowl- 
edge of phenomena in the atmosphere and space. The Administration 
shall provide for the widest practicable and appropriate dissemination 
of information concerning its activities and the results thereof ” 

— National Aeronautics and Space Act of 1958 
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